Medical image processing apparatus and medical image processing method

ABSTRACT

A medical image processing apparatus and a medical image processing method that can improve extraction accuracy of a tumor region included in a diagnosis target image are provided. The medical image processing apparatus configured to extract a predetermined region from a diagnosis target image includes: an organ extraction unit configured to extract an organ region from the diagnosis target image; and a tumor extraction unit generated by executing machine learning using a known tumor region included in each medical image group as teacher data and using an organ region extracted from the medical image group and the medical image group as input data. The tumor extraction unit is configured to extract a tumor region from the diagnosis target image using the organ region extracted from the diagnosis target image by the organ extraction unit.

CLAIM OF PRIORITY

The present application claims priority from Japanese Patent ApplicationJP 2021-074718 filed on Apr. 27, 2021, the content of which is herebyincorporated by reference into this application.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present invention relates to a medical image processing apparatusand a medical image processing method for extracting a tumor included ina medical image.

2. Description of the Related Art

A medical image capturing device that is typified by an X-ray computedtomography (CT) device or the like is a device that captures an image ofa morphology of a lesion or the like, and the obtained medical image isused for image diagnosis or treatment planning. In order to executeappropriate image diagnosis or treatment planning, it is important toclassify tissues and lesions with high accuracy.

JP-A-2018-175217 describes an image processing apparatus capable ofclassifying regions of tissues and lesions included in a medical imagewith high accuracy. Specifically, the image processing apparatusidentifies, using a determiner that executes machine learning using atomographic image as teacher data, a type of a tissue or a lesion towhich pixels of tomographic images in different cross-sectionaldirections belong, and re-identifies the type of a lesion or the like byevaluating a pixel common to a plurality of tomographic images. In thetomographic image, the type of a tissue or a lesion is known.

However, in JP-A-2018-175217, only a determiner that individuallyexecutes machine learning on the type of a tissue or a lesion, and it isinsufficient to extract a lesion such as a tumor with higher accuracy.

SUMMARY OF THE INVENTION

An object of the invention is to provide a medical image processingapparatus and a medical image processing method that can improveextraction accuracy of a tumor region included in a diagnosis targetimage.

In order to achieve the above object, the invention provides a medicalimage processing apparatus configured to extract a predetermined regionfrom a diagnosis target image. The medical image processing apparatusincludes: an organ extraction unit configured to extract an organ regionfrom the diagnosis target image; and a tumor extraction unit generatedby executing machine learning using a known tumor region included ineach medical image group as teacher data and using an organ regionextracted from the medical image group and the medical image group asinput data. The tumor extraction unit is configured to extract a tumorregion from the diagnosis target image using the organ region extractedfrom the diagnosis target image by the organ extraction unit.

The invention provides a medical image processing method for extractinga predetermined region from a diagnosis target image. The medical imageprocessing method includes: an organ extraction step of extracting anorgan region from the diagnosis target image; and a tumor extractionstep of extracting a tumor region from the diagnosis target image usingthe organ region extracted from the diagnosis target image in the organextraction step.

According to the invention, a medical image processing apparatus and amedical image processing method that can improve extraction accuracy ofa tumor region included in a diagnosis target image can be provided.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an overall configuration diagram of a medical image processingapparatus;

FIG. 2 is a diagram showing a functional block according to a firstembodiment;

FIG. 3 is a diagram showing an example of a processing flow forgenerating a tumor extraction unit;

FIG. 4 is a diagram showing an example of a processing flow according tothe first embodiment;

FIG. 5A is a diagram showing image capturing of a chest of a subject;

FIG. 5B is a diagram showing an example of a tomographic image;

FIG. 5C is a diagram showing an example of an extracted organ region;

FIG. 5D is a diagram showing an example of an extracted tumor region;

FIG. 5E is a diagram showing an example of an image obtained by fusingthe organ region and the tumor region;

FIG. 6 is a diagram showing an example of model configurations of anorgan extraction unit and the tumor extraction unit;

FIG. 7 is a diagram showing another example of the model configurationsof the organ extraction unit and the tumor extraction unit;

FIG. 8 is a diagram showing a functional block according to a secondembodiment;

FIG. 9 is a diagram showing an example of a processing flow according tothe second embodiment;

FIG. 10 is a diagram showing an example of a feature amount; and

FIG. 11 is a diagram showing an example of an input and output screenaccording to the second embodiment.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, embodiments of a medical image processing apparatus and amedical image processing method according to the invention will bedescribed with reference to accompanying drawings. In the followingdescription and the accompanying drawings, components having the samefunction and structure are denoted by the same reference numerals, andrepeated description thereof will be omitted.

First Embodiment

FIG. 1 is a diagram showing a hardware configuration of a medical imageprocessing apparatus 1. The medical image processing apparatus 1includes an arithmetic unit 2, a memory 3, a storage device 4, and anetwork adapter 5, which are connected via a system bus 6 so as totransmit and receive a signal. The medical image processing apparatus 1is connected to a medical image capturing device 10 and a medical imagedatabase 11 via a network 9 so as to transmit and receive a signal.Further, a display device 7 and an input device 8 are connected to themedical image processing apparatus 1. Here, “so as to transmit andreceive a signal” indicates a state of being able to transmit andreceive a signal to and from each other or from one to the otherregardless of whether the connection is electrically or optically wiredor wireless.

The arithmetic unit 2 is a device that controls operations ofcomponents, and is specifically a central processing unit (CPU), a microprocessor unit (MPU), or the like. The arithmetic unit 2 loads a programstored in the storage device 4 and data necessary for executing theprogram into the memory 3 and executes the program, and executes varioustypes of image processing on the medical image. The memory 3 stores theprogram to be executed by the arithmetic unit 2 and a progress ofarithmetic processing. The storage device 4 is a device that stores theprogram to be executed by the arithmetic unit 2 and the data necessaryfor executing the program, and is specifically a hard disk drive (HHD),a solid state drive (SSD), or the like. The network adapter 5 is usedfor connecting the medical image processing apparatus 1 to the network 9such as a local area network (LAN) , a telephone line, or the Internet.Various data to be processed by the arithmetic unit 2 may be transmittedto and received from the outside of the medical image processingapparatus 1 via the network 9 such as the LAN.

The display device 7 is a device that displays a processing result orthe like of the medical image processing apparatus 1, and isspecifically a liquid crystal display, a touch panel, or the like. Theinput device 8 is an operation device for an operator to give anoperation instruction to the medical image processing apparatus 1, andis specifically a keyboard, a mouse, a touch panel, or the like. Themouse may be another pointing device such as a trackpad or a trackball.

The medical image capturing device 10 is a device that captures atomographic image or the like of the subject, and for example, is anX-ray CT device, a magnetic resonance imaging (MRI) device, and apositron emission tomography (PET) device. The medical image database 11is a database system that stores the medical images such as thetomographic image captured by the medical image capturing device 10 anda corrected image obtained by executing image processing on thetomographic image.

A functional block diagram according to a first embodiment will bedescribed with reference to FIG. 2. Each function shown in FIG. 2 may beimplemented by dedicated hardware using an application specificintegrated circuit (ASIC), a field-programmable gate array (FPGA), orthe like, or may be implemented by software operating on the arithmeticunit 2. In the following description, a case will be described in whicheach function according to the first embodiment is implemented by thesoftware.

In the first embodiment, an organ extraction unit 201 and a tumorextraction unit 202 are provided. The storage device 4 stores adiagnosis target image or the like. The diagnosis target image is amedical image captured by the medical image capturing device 10 and is adiagnosis target. The diagnosis target image may be a tomographic imageor a volume image. Hereinafter, each component will be described.

The organ extraction unit 201 extracts an organ region from thediagnosis target image based on a pixel value of a pixel included in thediagnosis target image. The extracted organ region is classified into,for example, a heart, a lung, a body surface, or the like. The organextraction unit 201 may include, for example, a convolutional neuralnetwork (CNN) generated by executing machine learning using a largenumber of medical images including a known organ region as input dataand the known organ region as teacher data.

The tumor extraction unit 202 is generated by executing machine learningusing a known tumor region included in the large number of medicalimages as teacher data to extract a tumor region from the diagnosistarget image, and includes, for example, the CNN.

An example of a processing flow for generating the tumor extraction unit202 will be described for each step with reference to FIG. 3.

-   S301

Medical image groups including the known tumor region are obtained. Thatis, each pixel included in the medical images is assigned an identifierindicating whether the pixel is a tumor region.

-   S302

An organ region is extracted from each medical image group obtained inS301. That is, each pixel included in the medical images is assigned anidentifier indicating whether the pixel is an organ region, and anidentifier indicating which organ the pixel is further assigned to thepixel which is the organ region.

In the extraction of the organ region from the medical images, the organextraction unit 201 may be used. When the known organ region is includedin the medical images, an identifier of each pixel is used.

-   S303

The tumor extraction unit 202 is generated by executing machine learningusing the medical image groups obtained in S301 and the organ regionextracted in S302 as input data and the known tumor region included inthe medical images as teacher data. Specifically, a weight value of eachpath of an intermediate layer connecting an input layer and an outputlayer is adjusted such that output data output from the output layermatches the teacher data when the input data is input to the inputlayer.

As described above, the tumor extraction unit 202 is generated by thedescribed processing flow. Since the generated tumor extraction unit 202executes machine learning using not only the medical image groupsincluding the known tumor region but also the organ region extractedfrom each medical image group as input data, the tumor extraction unit202 includes, as knowledge, relation between the tumor region and theorgan region, for example, relative positional relation between thetumor region and the organ region.

An example of a processing flow for extracting the tumor region from thediagnosis target image according to the first embodiment will bedescribed for each step with reference to FIG. 4.

-   S401

The diagnosis target image that is a medical image of a diagnosis targetis obtained. The diagnosis target image is, for example, a tomographicimage obtained by capturing an image of the chest of the subject shownin FIG. 5A by the medical image capturing device 10, and is transmittedvia the network adapter or read from the storage device 4. FIG. 5B showsthe tomographic image of the chest. The obtained diagnosis target imageis not limited to one tomographic image, and may be a plurality oftomographic images, or a volume image.

-   S402

The organ extraction unit 201 extracts an organ region from thediagnosis target image obtained in S401. The extracted organ region isclassified into a heart, a lung, a body surface, and the like as shownin FIG. 5C.

-   S403

The tumor extraction unit 202 extracts a tumor region from the diagnosistarget image using the organ region extracted in S402. FIG. 5D shows thetumor region extracted from the tomographic image of the chest. Alongwith the extraction of the tumor region, the organ region extracted inS402 may be subdivided, and for example, a spine may be extracted. Asshown in FIG. 5E, the extracted tumor region and the organ region may befused.

As described above, according to the described processing flow, thetumor region can be extracted from the diagnosis target image. Inparticular, since the tumor extraction unit 202 includes relationbetween the tumor region and the organ region as knowledge, the tumorextraction unit 202 can extract the tumor region with high accuracy ascompared with a determiner that only executes machine learning using aknown tumor region as teacher data.

An example of model configurations of the organ extraction unit 201 andthe tumor extraction unit 202 will be described with reference to FIG.6. The organ extraction unit 201 and the tumor extraction unit 202 thatare generated by executing machine learning are configured based on theCNN. When three-dimensional CT data, which is the diagnosis targetimage, is input to the input layer of the organ extraction unit 201, anorgan region is extracted. The extracted organ region is incorporatedinto the tumor extraction unit 202, and is used when the tumorextraction unit 202 extracts a tumor region from the diagnosis targetimage. The extracted organ region may be incorporated into theintermediate layer of the tumor extraction unit 202, or may be input tothe input layer. Finally, the organ region extracted by the organextraction unit 201 and the tumor region extracted by the tumorextraction unit 202 are fused.

Another example of the model configurations of the organ extraction unit201 and the tumor extraction unit 202 will be described with referenceto FIG. 7. In the example in FIG. 6, a model of the organ extractionunit 201 and a model of the tumor extraction unit 202 are divided,whereas in the example in FIG. 7, the organ extraction unit 201 and thetumor extraction unit 202 are configured by one model, and an extractionresult of the organ region is incorporated via a layer inside a networkand used for the extraction of the tumor region. Since the size of themodels can be reduced according to the configuration shown in FIG. 7, itis easy to implement the medical image processing apparatus 1.

Second Embodiment

In the first embodiment, the tumor region is extracted from thediagnosis target image by the tumor extraction unit 202 that isgenerated by executing machine learning using not only the medical imagegroups including the known tumor region but also the organ regionextracted from each medical image group as input data. In a secondembodiment, calculation of a feature amount related to the extractedtumor region will be described. The feature amount related to the tumorregion can be used to support the image diagnosis and the treatmentplanning. Since the hardware configuration of the medical imageprocessing apparatus 1 according to the second embodiment is the same asthat according to the first embodiment, the description thereof will beomitted.

A functional block diagram according to the second embodiment will bedescribed with reference to FIG. 8. Similar to the first embodiment,each function shown in FIG. 8 may be implemented by dedicated hardwareusing an ASIC, an FPGA, or the like, or may be implemented by softwareoperating on the arithmetic unit 2. In the following description, a casewill be described in which each function according to the secondembodiment is implemented by the software.

In the second embodiment, similar to the first embodiment, the organextraction unit 201 and the tumor extraction unit 202 are provided, anda feature amount calculation unit 403 and a state determination unit 404are further provided. Hereinafter, the feature amount calculation unit403 and the state determination unit 404 that are added to theconfiguration according to the first embodiment will be described.

The feature amount calculation unit 403 calculates a feature amountrelated to the tumor region extracted from the diagnosis target image.The feature amount related to the tumor region includes a tumor propertyfeature amount that is a feature amount related to the tumor regionitself, a tumor-organ feature amount that is a feature amountrepresenting the relation between the tumor region and the organ region,and an intermediate layer feature amount that is a feature amount usedin an intermediate layer of the tumor extraction unit 202.

The state determination unit 404 determines a state of the tumor regionbased on the feature amount calculated by the feature amount calculationunit 403. The state of the tumor region includes a size, an infiltrationstage, classification, a development stage, and the like of the tumor.

An example of a processing flow for extracting a tumor region from adiagnosis target image and calculating the feature amount related to thetumor region according to the second embodiment will be described foreach step with reference to FIGS. 9. S301 to S303 are the same as thoseaccording to the first embodiment, and thus the description thereof willbe omitted.

-   S904

The feature amount calculation unit 403 calculates the feature amountrelated to the tumor region extracted in S303, that is, a tumor propertyfeature amount, a tumor-organ feature amount, and an intermediate layerfeature amount.

An example of the feature amount calculated by the feature amountcalculation unit 403 will be described with reference to FIG. 10. Thetumor property feature amount is, for example, a value indicating a sizeof each tumor region, a value indicating a shape, a value calculatedbased on a luminance histogram, or Radomics. The value indicating thesize of the tumor region includes a volume and a maximum diameter. Thevalue indicating the shape of the tumor region includes an index valueindicating circularity or the presence or absence of a cavity. The valuecalculated based on the luminance histogram includes uniformity of aluminance value and a half-value width of a maximum peak. The tumorproperty feature amount is used to support the image diagnosis.

The tumor-organ feature amount is, for example, a distance between atumor region and an organ region, an index value indicating the presenceor absence of adhesion, or an index value indicating the presence orabsence of infiltration. A position, a distribution, the number, and thelike of the tumor in an image capturing part may be included in thetumor-organ feature amount. The tumor-organ feature amount is used tosupport the image diagnosis and the treatment planning.

The intermediate layer feature amount is a value or the like indicatinginformation shared by the tumor region and the organ region. Theintermediate layer feature amount is used for metastasis learning andprediction of a therapeutic effect.

The description returns to FIG. 9.

-   S905

The state determination unit 404 determines the state of the tumorregion based on the feature amount calculated in S904. S905 is notessential.

According to the processing flow described above, the tumor region isextracted from the diagnosis target image, the feature amount related tothe tumor region is calculated, and the state of the tumor isdetermined. In the second embodiment, similar to the first embodiment,the tumor region is extracted with high accuracy. The feature amountrelated to the tumor region calculated by the feature amount calculationunit 403 is used to support the image diagnosis and the treatmentplanning.

An example of an input and output screen according to the secondembodiment will be described with reference to FIG. 11. An input andoutput screen 500 shown in FIG. 11 includes an axial image display part511, a sagittal image display part 512, a coronal image display part513, a three-dimensional image display part 514, a feature amountdisplay part 515, and a determination result display part 516.

The axial image display part 511 displays an axial image of thediagnosis target image. The sagittal image display part 512 displays asagittal image of the diagnosis target image. The coronal image displaypart 513 displays a coronal image of the diagnosis target image. A lineindicating a contour of the extracted organ region or tumor region maybe superimposed and displayed on the axial image, the sagittal image, orthe coronal image. The three-dimensional image display part 514 displaysa three-dimensional image of the diagnosis target image.

The feature amount display part 515 displays the feature amountcalculated by the feature amount calculation unit 403. The featureamount display part 515 shown in FIG. 11 displays that a tumor size is28 mm, a side edge is a leaflet, there is no air inclusion, theluminance distribution is solid, there is adhesion with the lungs, thereis no adhesion with the heart, and the like. The determination resultdisplay part 516 displays the state of the tumor region determined bythe state determination unit 404. The determination result display unit516 shown in FIG. 11 displays that a stage is IIIb and theclassification is large cells.

The input and output screen 500 includes an image selection button 521,an region extraction button 522, an region selection button 523, anregion edit button 524, a feature amount setting button 525, a resultoutput button 526, and a state determination button 527.

The image selection button 521 is used to select a diagnosis targetimage. The selected diagnosis target image is displayed on the axialimage display part 511, the sagittal image display part 512, the coronalimage display part 513, and the three-dimensional image display part514.

The region extraction button 522 is used to extract an organ region or atumor region from the diagnosis target image. The extracted organ regionand tumor region are superimposed and displayed on the axial imagedisplay part 511, the sagittal image display part 512, and the coronalimage display part 513.

The region selection button 523 is used to select the extracted organregion and tumor region. The region edit button 524 is used to edit theextracted organ region and tumor region. The selection and the editingof the organ region and the tumor region are executed in the axial imagedisplay part 511, the sagittal image display part 512, and the coronalimage display part 513.

The feature amount setting button 525 is used to set the feature amountrelated to the tumor region. The set feature amount is calculated by thefeature amount calculation unit 403 and displayed on the feature amountdisplay part 515.

The result output button 526 is used to output an extraction result ofthe organ region and the tumor region and a calculation result of thefeature amount from the medical image processing apparatus 1.

The state determination button 527 is used to determine the state of thetumor region. The state of the tumor region is determined by the statedetermination unit 404 and displayed on the determination result displaypart 516.

By using the input and output screen 500 shown in FIG. 11, the operatorcan select a diagnosis target image or check the tumor region extractedfrom the diagnosis target image together with the feature amount relatedto the tumor region.

As described above, a plurality of embodiments of the invention havebeen described. The invention is not limited to the above embodiments,and can be embodied by modifying constituent elements without departingfrom a spirit of the invention. A plurality of constituent elementsdisclosed in the above embodiments may be appropriately combined.Further, some constituent elements may be deleted from all theconstituent elements shown in the above embodiments.

What is claimed is:
 1. A medical image processing apparatus configuredto extract a predetermined region from a diagnosis target image, themedical image processing apparatus comprising: an organ extraction unitconfigured to extract an organ region from the diagnosis target image;and a tumor extraction unit generated by executing machine learningusing a known tumor region included in each medical image group asteacher data and using an organ region extracted from the medical imagegroup and the medical image group as input data, wherein the tumorextraction unit is configured to extract a tumor region from thediagnosis target image using the organ region extracted from thediagnosis target image by the organ extraction unit.
 2. The medicalimage processing apparatus according to claim 1, wherein the organextraction unit is generated by executing machine learning using theknown organ region included in each medical image group as teacher dataand using the medical image group as input data, and the input data usedfor generation of the tumor extraction unit includes the organ regionextracted from the medical image group by the organ extraction unit. 3.The medical image processing apparatus according to claim 1, furthercomprising a feature amount calculation unit configured to calculate afeature amount related to the tumor region extracted by the tumorextraction unit.
 4. The medical image processing apparatus according toclaim 3, wherein the feature amount calculated by the feature amountcalculation unit includes a tumor-organ feature amount that is a featureamount representing relation between the organ region extracted by theorgan extraction unit and the tumor region extracted by the tumorextraction unit.
 5. The medical image processing apparatus according toclaim 4, wherein the tumor-organ feature amount includes a distancebetween the organ region and the tumor region, presence or absence ofadhesion between the organ region and the tumor region, and presence orabsence of infiltration of the tumor region into the organ region. 6.The medical image processing apparatus according to claim 3, wherein thefeature amount calculated by the feature amount calculation unitincludes a tumor property feature amount that is a feature amountrelated to the tumor region itself extracted by the tumor extractionunit.
 7. The medical image processing apparatus according to claim 6,wherein the tumor property feature amount includes a histogram of asize, a shape, and a pixel value of the tumor region.
 8. The medicalimage processing apparatus according to claim 3, further comprising astate determination unit configured to determine a state of the tumorregion based on the feature amount calculated by the feature amountcalculation unit.
 9. A medical image processing method for extracting apredetermined region from a diagnosis target image, the medical imageprocessing method comprising: an organ extraction step of extracting anorgan region from the diagnosis target image; and a tumor extractionstep of extracting a tumor region from the diagnosis target image usingthe organ region extracted from the diagnosis target image in the organextraction step.